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EXECUTIVE SUMMARY 

The National Post-School Outcomes Center (NPSO), in conjunction the 
National Association of State Directors of Special Education (NASDSE) and 
the Regional Resource Centers (RRCs), held a series of conference calls with 
states to assess their needs for technical assistance on the new requirements 
for collecting and reporting post-school outcome data for students with 
disabilities as required by the U.S. Department of Education's Office of 
Special Education Programs (OSEP). The major concerns expressed by the 
states had to do with data collection, including sampling techniques that 
produce reliable and valid post-school outcome data. States were also 
concerned about the costs involved, the burden placed on local education 
agencies and a need to learn about practices currently in place that are 
producing good data. Additionally, states expressed concern that key terms, 
such as "rigorous targets" and "meaningful employment" have not been 
defined by OSEP. 

This paper summarizes the issues raised by the states on the conference 
calls. The states' comments suggest areas for further policy and practice 
considerations, including the following: 

1. A "Frequently Asked Questions" (FAQ) or other document from OSEP 
(perhaps prepared by NPSO) that addresses the definitional 
questions and some of the sampling questions. 

2. A document sharing "best practices" from those states that have 
demonstrated success in collecting post-school outcome data. 

3. A mechanism for sharing/ comparing among states of similar size or 
states with similar concerns. (This could be facilitated at the March 
NPSO conference.) 

4. Conference calls with "experts" in developing post-school outcome 
questionnaires and sampling procedures. 

5. Issues of conflict between federal and state laws regarding data 
collection need to be recognized and addressed by OSEP (e.g., 
providing added flexibility). 



INTRODUCTION 

The National Post-School Outcomes Center (NPSO), in conjunction with the 
National Association of State Directors of Special Education (NASDSE), 
conducted a series of conference calls in August 2005 with state directors of 
special education, state transition coordinators and RRC transition 
coordinators. The purpose of the calls was to share information about the 
new NPSO Center and to solicit comments from the states about their 
concerns, challenges, and issues involved in collecting meaningful post- 
school outcome data and suggestions as to what kinds of technical assistance 
the new center might provide. 
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The calls were held on August 2, 2005 and August 15, 2005. The following 
states participated in the calls: Alabama, Alaska, Arizona, California, 
Connecticut, Florida, Georgia, Illinois, Kansas, Louisiana, Maine, Maryland, 
Michigan, Minnesota, Missouri, Montana, New Jersey, New Mexico, New York, 
North Dakota, Oklahoma, Oregon, Pennsylvania, Rhode Island, South 
Dakota, Utah, Wisconsin, Wyoming and the Virgin Islands. The following 
regional resource centers also participated in the calls: Mid South (MSRRC); 
North East (NERRC); Mountain Plains (MPRRC); North Central (NCRRC); 
South East (SERRC) and Western (WRRC). 

The calls were each structured around a series of questions, which were 
preceded by an overview of the Center and a review of materials that the 
Center has produced to date or is in the midst of developing. These 
materials/activities are: 

■ sessions at the upcoming OSEP Summer Institute; 

■ a checklist, case study and sampling paper all related to the State 
Performance Plans (SPP); 

■ an application template for General Supervision Enhancement Grants 
(GSEGs); 

■ the Community of Practice on post-school outcomes; and 

■ a possible national conference in late February 2006. 

(NOTE: All documents produced by the Center are now available on the 
Center's website at http://psocenter.org.) 

The remainder of this report summarizes the responses to the questions 
posed during the conference calls. 



SUMMARY OF COMMENTS AND RESPONSES TO QUESTIONS 

Comments on the conference calls clustered around several areas, including 
the following: (1) definitions; (2) concerns over cost and burden of data 
collection; (3) data collection, including validity of data and coordination with 
other agencies; (4) sampling; (5) follow-up with students after they leave 
the system; and (7) best practices. Specific comments in each of these 
areas are listed below. Unless otherwise noted, each of these comments was 
posed by one state. 

Definitions 

■ How is "enrolled in program" defined? 

■ What are "rigorous targets" (1 state) and how should they be connected 
to measurable post-schools goals and post-school outcomes? (1 state) 

■ How can differing definitions from different agencies be integrated and 
useful for purposes of collecting post-school outcome data? 

■ How is "competitively employed" defined? 
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Concerns over cost and burden of data collection 

■ There is concern about the reporting burden on the local education 
agencies (LEAs). 

■ There is a need for tools that can be used by LEAs and other local 
agencies to ease the data collection burden for them (because the data 
collected at the state level may not be able to be disaggregated at the 
local level and thus may be of little value to local agencies). 

■ There was concern expressed over the cost of collecting data— both in 
terms of personnel time and programming costs— and interest in 
strategies for streamlining data collection, (several states) 

Data collection, including validity of data 

■ One state (OR) noted that state law requires the state education agency 
(SEA) to inform LEAs of any new data collection one year prior to actually 
collecting the data. These deadlines cannot be met with respect to the 
SPP data collection requirements. 

■ One state questioned how it could require LEAs to collect data on 
students once they have left the system. (This issue relates to the 
burden being put on LEAs as well.) 

■ One state wanted to know how to ensure that it has collected sufficient 
data to report. 

■ One state expressed interest in designing a system that will include goals 
and outcomes upfront so that the data collected can be used to 
document progress. 

■ Six states specifically asked questions about how to collect the data, how 
to ensure its validity and reliability and what to do with it once they 
collected it. For example, how can the data be used to pinpoint where 
help is needed and what kind of help is needed. How can the data be 
made meaningful across LEAs that are very different in size— ranging 
from 1-2 exiting students per year to thousands and ranging from urban 
to extremely rural? One state commented that just because someone is 
"involved" in a program doesn't mean that the person is attending the 
program and that doesn't equate with success or completion of the 
program. This state asked "how many days should an individual be 
employed for it to equate with success and what kind of employment is 
considered adequate or gainful?" 

■ Two states wanted to know how to use the data to improve transition 
services. One of these states expressed concern about establishing 
effective programs based on what they are learning about students one 
year out of school. 

■ Two states specifically expressed concern about the difficulties associated 
with sharing data across agencies and one of them wondered if this data 
collection could be integrated into any other data sets that other state 



4 




and/or local agencies are currently collecting. A third state noted the 
importance of ensuring a cross-agency perspective in the data collection. 

Sampling 

■ One state expressed concern that sampling would not be useful in 
holding LEAs accountable for their outcomes. One state (Wl) noted that 
it samples the entire state every other year and has created a website so 
LEAs can do their own local outcomes using the same procedures that 
the state uses. The state does a random sampling of LEAs, not of 
individuals. 

■ One state expressed concern that it is difficult to "sell" sampling to LEAs 
because it generates a lot of paperwork and it is difficult for LEAs to see 
the value in sampling. 

■ One state asked how many students need to be sampled. 

Follow up 

■ At least five states expressed concerns related to following up with 
students. One state wanted to know how to track down students. (NC 
indicated that it has integrated a post-school data collection piece into an 
exit document so that when students respond to questions about exiting, 
the data will be entered and the students will be followed up on at one 
and three years post exiting.) 

■ One state commented that it formerly used social security numbers to 
collect follow-up information, but can no longer do so and expressed 
concern that follow-up would be more difficult as a result. 

■ One state expressed concern that it would be difficult to collect 
information on dropouts because it would be difficult to locate them and 
wondered if this is required by the SPP. 

■ One state wanted to know how to track down students— by sending a 
letter or calling them. 

■ One state expressed interest in the development of a "script" that could 
be used by LEAs when contacting students or their parents for follow-up. 

■ One state expressed concern about how to track mobile students in the 
16-21 age range who move frequently back and forth from school to 
work and from job to job. 

■ One state expressed concern about the low response rate from surveyed 
students and asked for help on ways to improve this. 

Best Practices 

■ Several states agreed that it would be helpful if NPSO could collect 
information on what states are currently doing and whether they think it 
has been effective or not. (It was noted that the state profiles that NPSO 
plans to develop would address this.) 
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NPSO 

The Center is considering developing state profiles about post-school 
outcome data systems. The Center will compile a common set of information 
across all of the states and create some mechanisms for states to be able to 
find out, for example, which states contract with universities, which states 
contract with other organizations, etc. The purpose is to get a common 
picture about how all of the states gather and use information, who their 
partners are, etc. These might include, for example, post-school outcome 
data collected, timing, description of sampling procedures, reporting and use 
of data for improvement. Other information states would like to see included 
in the profile: 

■ all of the above examples; 

■ leveraging across offices; 

■ the instruments used; 

■ posting frequently asked questions; 

■ demographic information— including size, population, number of students 
exiting per year; 

■ whether states used sampling or attempted to contact all exiting 
students; 

■ metrics on cost of outsourcing; and 

■ identifying response rates. 

MISCELLANEOUS COMMENTS 

■ How does the summary of performance connect, or should it connect, to 
the post-school outcomes (e.g., linking mechanisms, linking questions or 
linking components)? 

■ Should SEAs and LEAs consider beginning to collect data when students 
are in the 9th grade, to make better use of it and make it easier to track 
students once they exit? 

■ How can we build a data system that is attractive and marketable to 
other customers (e.g., other agencies)? 

■ It is important to leverage these requirements with other education 
initiatives such as high school reform. 

■ Comments on existing data collection systems: 

• Montana has a web- based system for LEAs to report exiting 
data. 

• New Mexico contracts with an outside entity to compile and 
analyze data— they have trained interviewers. 

• Alabama contracts with Auburn University. 

■ Post state contacts on the Center's website so states can contact each 
other. 

■ What states already have good systems in place for collecting data? 

(Jane Falls provided the following examples. She does not intend it to be 
an exhaustive list.) 
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• Alabama has a sophisticated system and is working on 
increasing the response rate. 

• Idaho uses a web- based exit survey. It contracts with Life Track 
in Clarkson, WA, to do follow-up and interview students one, 
three and five years out. It also tries to contact all students— 
about 1,200 graduates per year. 

• Washington State has 96% of its LEAs engaged and has about a 
75-78% response rate and has a mechanism in place to feed 
information back to the LEAs. 



POLICY/PRACTICE IMPLICATIONS 

The issues and concerns raised by the states suggest that technical 
assistance by the Center should focus on the "nitty-gritty" of data collection, 
sampling, how to ensure the validity and reliability of the data collected and 
how to use the data once it is collected. In addition, the issues raised 
suggest specific policy and guidance issues that could be addressed by some 
of the following: 

1. A "Frequently Asked Questions" (FAQ) or other document from QSEP 
(perhaps prepared by NPSQ) that addresses the definitional 
questions and some of the sampling questions. 

2. A document/website summaries/state profiles that share "best 
practices" from those states that have demonstrated success in 
collecting post-school outcome data. 

3. A mechanism for sharing/comparing among states of similar size or 
states with similar concerns. (This could be facilitated at the March 
NPSQ conference.) 

4. Conference calls with "experts" in developing post-school outcome 
questionnaires and sampling techniques. 

5. Issues of conflict between federal and state laws regarding data 
collection need to be recognized and addressed by QSEP (e.g., 
providing added flexibility). NASDSE could play a role in collecting 
this information and working with QSEP on areas of clarification and 
flexibility. 

Report compiled by: 

Nancy Reder 
NASDSE 
November 2005 
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